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Motivated by the neutrino data, an extension of the Standard Model with three Higgs-boson 
doublets has been proposed. Imposing an 0(2) (g) 7 L -2 family symmetry, a neutrino mixing matrix 
with 6^23 = 7 r /4 and 613 = 0 appears in a natural way. Even though these values for the mixing matrix 
do not follow the recent experimental constraints, they are nevertheless a good approximation. We 
study the Higgs potential of this model in detail. We apply recent methods which allow for the 
study of any three-Higgs-boson doublet model. It turns out that for a variety of parameters the 
potential is stable, has the correct electroweak symmetry breaking, and gives the correct vacuum 
expectation value. 


1. THE 0(2) (g) Z 2 MODEL 

The experimental neutrino mixing data show that the 
neutrino mixing is very different from the quark mixing. 
In the usual parametrization of the neutrino mixing ma¬ 
trix (see for instance mi experimental data suggest that 
the angle ^13 is small (but nonzero), and 6*23 is close to 
7 r /4 0. 

There is lot of effort spent on finding an organizing 
principle for the flavor puzzle. A general approach is 
to study finite subgroups of SU(3) which have an irre¬ 
ducible triplet representation; see for instance and 
references therein. Examples of subgroups with triplet 
representations are S4, the group of permutations of 4 
elements, with two singlet, one doublet, and two triplet 
representations (see for instance HI); or A4, the group of 
even permutations of 4 elements which has also a triplet 
representation besides singlet representations (see for in¬ 
stance Hli); or A(27), with 27 elements and two triplet 
representations (see for instance [7]). 

In contrast, here we want to study in detail the Higgs 
potential of a three Higgs-boson-doublet model which im¬ 
poses a 0(2) X symmetry - without any irreducible 
triplet representation. Let us closely follow the motiva¬ 
tion of [8]. The starting point is a neutrino mass matrix 
which is symmetric in generations two and three, 

(x y y\ 

M,, = \y z u; . (1.1) 

\y w z j 

This mass matrix may be diagonalized as usual, that is, 

MylJ = where U is the neutrino 

mixing matrix and the neutrino masses. 
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Expressing the mixing matrix U in terms of the usual 
parametrization [T], we get in particular ^13 = 0 and 
623 = 7r/4. Even though the experimental results are 
not in exact agreement with these values, in particu¬ 
lar Oi 3 is nonzero, they at least appear to be approxi¬ 
mately fulfilled. We note that deviations from these ap¬ 
proximate values for the mixing angles could arise by 
soft-symmetry-breaking terms or beyond-leading-order 
effects; see for instan ce th e study in [S]. 

The mass matrix (1.1) may be generated by the in¬ 


troduction of three Higgs-boson doublets tpi, i = 1,2,3, 
and a symmetry 0(2) x W 12 = S2 x S2 x 17(1), where 
all elementary particles are assigned to an appropriate 
transformation behavior on this symmetry. The reflec¬ 
tion symmetry TL!^, is responsible for the \x-t symmetry 
of (O): 


^2 ■ ^ 21rL, Ur <->■ Tr, <->■ y^Ri (j>l ^ 4>2 


( 1 . 2 ) 


Here and DrL denote the left-handed 517(2) lep¬ 
ton doublets, v^R and VtR the right-handed neutrinos, 
and all remaining fields transform trivially under the ^2 
symmetry. The S2 symmetry is given by a sign change. 


^2 : '^eR —t —l^eR, ^tR “t —^tR, 

GR -CR, (P 3 -t -(P 3 , (1-3) 

with VeR, RfiR’ RtR. the right-handed neutrinos, cr the 
right-handed electron, and all other fields unchanged un¬ 
der this S2 symmetry. Eventually the assignment with 
respect to the phase symmetry 17(1) is 


17(1): 


A —^ C^X 

DfiLMRT^uR DtL, UR,RtR 

Pi 

<72 

6 

1 -1 

2 

-2 


(1.4) 


with X one of the helds on the right-hand side of the 
table transforming as X —e*® A with the corresponding 
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phase 9 given explicitly in the table. All other fields 
transform trivially. 

By virtue of these symmetries - besides the electroweak 
SU{2)l X U{\)y symmetry - there appear in particular 
the invariant Yukawa couplings 

■^Y = -Va + DrL^2TR) + h.c. (1.5) 

The most general potential for the three Higgs-boson 
doublets 1 ^ 1 , {p 2 , reads 


^o( 2 )xZ 2 = 

+ Mm + ai{iplipsf 

+ a2<M3<M3 

+ 03 ( 95^1 • + ^ 1^2 ■ 

+ a4<M3‘Mi • ‘M3<^2 + aXifl^pz ■ 

+ 05 {{‘Pi'fiT + 

+ 06(^1 (Ml •‘M2</^2 + a7<Ml'^2 •‘M2‘M1) (1-6) 

where the t erm /im(<Mi‘M 2 + <M 2 '^i) breaks the U{1) sym¬ 
metry (1.4) explicitly but softly, since this is a quadratic 
term. For a nonvanishing parameter fi„i in this way ad¬ 
ditional Goldstone bosons are avoided, which otherwise 
would appear by spontaneous symmetry breaking of the 
U{1) symmetry. This potential has nine real parameters 
and one complex parameter 04 , corresponding to eleven 
real parameters in total. 

Now we want to discuss stability, stationarity, and elec¬ 
troweak symmetry breaking of this model. Of course 
only a model with a stable potential, having the correct- 
electroweak-symmetry-breaking behavior and the cor¬ 
rect vacuum expectation values is physically acceptable. 
These obvious constraints restrict the parameter space of 
the potential. Here, we focus on the Higgs potential and 
not on any further experimental limits. For instance, the 
expressions for the oblique parameters S, T, U are avail¬ 
able for any nHDM [10] and may be compared to the 
electroweak precision data. 

Since the 0(2) ® S 2 model is a 3HDM we encounter 
in this model four charged Higgs bosons and in total five 
neutral Higgs bosons. We expect a changed phenomenol¬ 
ogy of this model compared to the Standard Model. Of 
course, the detection of any further Higgs boson would 
be a clear signal for a model beyond the Standard Model. 


Depending on the Yukawa coupling strength in (1.5) we 
have for instance the signature of the production of a 
charged Higgs boson with subsequent decay into a muon 
and a muon-neutrino. The potential itself is in princi¬ 
ple detectable via its trilinear and quartic Higgs self¬ 
couplings. We leave this investigation for future work 
and focus here on the study of the Higgs potential with 
respect to stability, electroweak symmetry breaking and 
the global minimum. 


Even though the potential (1.6) appears to be rather 


involved we will see that it is indeed accessible in the bi¬ 
linear approach [IIHISj. In this approach gauge degrees 
of freedom are avoided systematically. Moreover, the cor¬ 
responding equations for stability and stationarity sim¬ 
plify in particular, the degree of systems of equations is 
lowered. Recently, the bilinear approach for the study of 
stability, stationarity, and electroweak symmetry break¬ 
ing has been extended to the study of any 3HDM [14j . 
which we now briefly review. 

The scalar pr oduc ts of the type i,j G {1,2,3}, 

in the potential (1.6) may be arranged in a 3x3 matrix 


K = (pl‘P2 ‘pI<P2 A‘^2 ■ 
V‘m1v53 4<M3 

By the introduction of the bilinears, 

Ko,=KX = tr{KK), a = 0,...,l 


(1.7) 


( 1 . 8 ) 


with Aq the 3x3 Gell-Mann matrices, the following re¬ 
placements can be made in the potential: 

. Kn K 3 Ks t 1 . 

^ ^ ^ = 2 + 

t t Ko Ks Ks 

(1.9) 

Av’a = ^ (^6 + iKj) , ^ 

Gomparing the potential, written in terms of bilinears 

V = + a,/3 = (0,...,8), (1.10) 

with the general form of the potential, we find the new 
parameters 
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iU = 



^J.0 + 2^12 n n n n n n ^12 

j A^m: 0; 0; 0; Oj 0; 


Mo 


76 
( ' 


1 

4 


g (oi + 202 + 205 4" ^e) 
0 
0 
0 
0 
0 
0 
0 

V^( — Ol — 7 + 05 + 


73 

0 

07 

0 

0 

0 

0 

0 

0 

0 


0 

0 

07 

0 

0 

0 

0 

0 

0 


0 

0 

0 

2o5 — 06 
0 
0 
0 
0 
0 


Obviously, all parameters are real in terms of bilinears. 
We note that there is a one-to-one correspondence be¬ 
tween the Higgs-boson doublets and the bilinear matrix 
K = \l 2 Ka\a with rank smaller or equal to two - except 
for irrelevant gauge degrees of freedom; see [T^ . 

Supposing the potential is bounded from below, at the 
global minimum, or the degenerate minima, the gradient 
of the potential has to vanish. The corresponding equa¬ 
tions may be used to fix some of the parameters. In order 
to obtain these equations we start with the parametriza- 
tion of the three Higgs-boson doublets with the same 
hypercharge y = 4 - 1 / 2 , in a particular gauge. 


^ V;^7i/2 + ^^i°/27) + *^?/27)) 


The derivatives of the potential (1.6), inserting (1.12) 
with respect to the fields at the vacuum, that is, for van¬ 
ishing fields give the nontrivial conditions 


Mo = “ 2 ^“^ “3)(7 4-u|) - aivl - Re(a4)uiU2, 

M12 = -^(«2 + az)vl - 05(7 + I’D, 

Mm = ^ ((205 - 06 - 07)uiU2 - Re(a4)u3) , 

Im(a 4 ) • U 2 U 3 = 0 , 

Im(a 4 ) • U 1 U 3 = 0 . 

(1.13) 


For nonvanishing vacuum expectation values, the last 
two equations immediately dictate that 04 has t o be 
real. Eventually, by means of the equations (1.13) the 
quadratic parameters mO) Mi 2 ) Mm may be expressed by 
the quartic parameters and the three vacuum expectation 
values vi, V 2 , V 3 . Further, the vacuum expectation values 


0 

0 

0 

0 

Vs/- 
3 V 

-01 - f 

4“ 05 -4 ^)'' 

0 

0 

0 

0 


0 


0 

0 

0 

0 


0 


0 

0 

0 

0 


0 


03 

0 

Reo4 

Ima4 


0 


0 

03 

Imo4 

—Reo4 


0 


Rea4 

Imo4 

03 

0 


0 


Imo4 

—Rea4 

0 

03 


0 


0 

0 

0 

0 

|oi 

- |o2 -4 

|o5 -4 |o6 / 


( 1 . 11 ) 


are restricted with view on the Yukawa couplings (1.5), 
that is, the ratio of the vacuum expectation values vi and 
V 2 has to be vi/v 2 = at tree-level accuracy. In 

addition, the vacuum expectation value 


uq = ~ 246 GeV (1-14) 

is given by the electroweak precision data. Therefore, all 
quadratic parameters follow from the quartic parameters 
and one free vacuum expectation value, say U 3 . There¬ 
fore, it appears reasonable to start with the following set 
of parameters, 


ai) 02 , osj Re(a 4 ), 05 , 06 , 07 , U 3 , 

Vo ~ 246 GeV, ui/u 2 = m^/nir- 


(1.15) 


Note that the tadpole conditions (1.13) only ensure that 
there is at least one stationary solution. By no means 
does this guarantee that the corresponding potential is 
stable and has a global minimum with the correct par¬ 
tially broken electroweak symmetry. 


2. STABILITY AND ELECTROWEAK 
SYMMETRY BREAKING IN THE 0(2) (g) Ka 
MODEL 


In this section we analyze the potential of the 0 ( 2 )( 8 )S 2 
model while varying two of its parameters. Starting with 
the paramet ers (1.15 ) we fix the quadratic parameters moj 
M12, Mm by (1.13). Quantitatively, we choose the quartic 
parameters motivated by the central point given in [5] 
with a variation of the two parameters oi G [ 0 ,5] and 
02 S [—3, 3] in steps of 0.2: 


oi G [0,5], 02 G [—3,3], 03 = —5, 04 = —0.0474, 

05 = 1.5, 06 = 2, 07 = 3, ( 2 . 1 ) 

vjv 2 = 105.66/1776.82, V 3 = uo/72. 
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with the masses of the muon, and tan, m,-, given in 
[I] . The central point in particular passes the electroweak 
precision observables - for details, see [S]. 

First, we study the stability of the potential. There¬ 
fore, we separate the potential into the quadratic and 
quartic parts V = K 0 J 2 + KqJ^, with J 2 and J 4 given 
by 


J 2 {ka) 

Ji{ka) 


flQ + 2^12 f /il 2 — ^J -0 


V 6 


V 73 


ks T lJ>mk \, 


— g(ai + 2 a 2 -l- 205 -|- og) 


H-—2ai — 02-1- 205 -|- 05)^8 H—^(^1 T ^2) 

3v 2 4 

+ ^(205 — ae ) k ^ + ~^ ik 4 + ^5 + “b ^7) 

+ ^^^(A:4fcg - k^kr) + + fcgfce) 


+ Y 2 ~ ^6)ksi 


( 2 . 2 ) 


with the vector components ka = Ka/Kg, a = 1,...,8 
defined for Kq ^ 0. For Kq = 0 the potential vanishes. 
The stationary points of J 4 {ka) corresponding to a matrix 
K with rank 2 are obtained from 


respectively, in the potential (1.6). Therefore it is 
evident that the potential is unstable for small values of 
Oi and too negative values for 02 - 


Having determined parameter sets giving a stable po¬ 
tential we proceed with the study of the stationary points 
in these cases. We systematically look for all stationary 
points of the potential. To this end we have to solve the 
following systems of polynomial equations, corresponding 
to solutions which break electroweak symmetry partially 
(conserving the elecromagnetic 17(1 )em symmetry), and 
solutions which break the electroweak symmetry com¬ 
pletely. 

The stationary solutions with full electroweak sym¬ 
metry breaking, corresponding to stationarity matrices 
K = KaXal^ of rank 2 are obtained from 




V{Ko,...,Ks) 


2 Kl - KaKa > 0 , 

det(i7) = 0 , 


i7o > 0 . 


u det(i7) 


= 0 , 


( 2 . 6 ) 


The stationary solutions with partial electroweak sym¬ 
metry breaking, corresponding to stationarity matrices 
K = KaXal2 of rank 1 are obtained from 


ki,...,ks 


J4{ka) -u{ det(\/77l3 + kaXa) 


= 0 , 


det(72/3l3 -l- kaXa) — 0, 

2 - kaka > 0 , 


(2.3) 


and the stationary points corresponding to a matrix 77 
with rank 1 are obtained from 


J4{ka{w\w) — U (w^w — 1) ] = 0, 


w^w — 1 = 0 

where we express the vector components ka by 


(2.4) 


ka{w\w) = 


-W^'XaW 


(2.5) 


and m, are three-component complex vectors. If we 
have for all real solu tions kg o f the systems of polyno¬ 
mial equations (2.3) and (2.4) J 4 {ka) > 0, or at least 
J 4 (ka) = 0 but then in addition J 2 (fco) > 0 , the poten¬ 
tial is stable. In other words, if there is for a given initial 
parameter set one solution with J 4 (ka) < 0 or J 4 {ka) = 0 
but in addition J 2 (fca) < 0 the potential is unstable. The 
unstable cases for the variation of parameter s (| 2 . 1 [ ) are 
denoted by the larger full disks (blue) in Fig. [H For all 
other values of parameters the potential is stable. 

Let us note that the quartic parameters oi and 02 ap¬ 
pear as coefficients of Lp\Lp^{ip\ipi -\- Lp\ip 2 )^ 


"^■w^,Ko\y{Ka{KQ,w\w)) - u{w'^w - 1 )] = 0 , 

w^w — 1 = 0, (2-7) 

77o >0 

where we express the bilinears 77„ in terms of Kq and 
the three component complex vectors w and w\ 


Ka{Ko, w^w) = ^^Kow''XaW, o = 0,..., 8. (2. 


8 ) 


In (2.6) and (2.71, u is a Lagrange multiplier, respectively. 


In addition there is always a solution for a vanish¬ 
ing potential, corresponding to an unbroken electroweak 
symmetry. 

The global minimum, that is, the vacuum, is given by 
the stationary point or points with the deepest potential 


value. In case this solution originates from the set (2.7), 


that is, when we do have electroweak symmetry break¬ 
ing SU{2)l X U{ 1 )y —t 17(l)em , we can directly cal¬ 
culate the vacuum-expectation value of this minimum, 


yg = VGKq, and verify that it coincides with (1.14). 


Depending on the variation of the parameters Oi and 
02 we detect the viable global minima. These cases are 
marked by little (green) dots in Fig. In case the deep¬ 
est potential value does not correspond to the correct 
electroweak symmetry breaking or does not give the cor¬ 
rect vacuum expectation value, these parameter points 
are denoted by a circle (red) in Fig. 

As we can see by the scattering of points, we typically 
find valid parameters for oi and 02 not too small. How¬ 
ever, the pattern of valid points appears very sensitive to 
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FIG. 1: Stability and stationarity solutions of the 3HDM Higgs potential, varying the two quartic parameters a\ and a 2 of 
the potential ( |1.6[ ). The other quartic parameters are set to 03 = —5, 04 = —0.0474, 05 = 1.5, ae = 2, 07 = 3. The quadratic 
parameters are fixed by the equations the vacuum expectation value vo, and the ratio ui/u 2 = nifilmr as well as 

V 3 = Vo/y/2. The larger full circles (blue) show points where the potential is unstable. The open circles (red) show parameters 
where no correct electroweak symmetry breaking appears or the vacuum expectation value Vo is unequal to 246 GeV. Finally, 
the small dots (green) have a viable global minimum corresponding to the correct vacuum expectation value. 


the parameter values. This, of course, is a consequence 


of the rather involved potential (1.6). 


Eventually, let us remark on the technical aspects to 
solve the rather involved systems of equations - on the 
one hand for the study of stability (2.3), (2.4), and on 


the other hand for the study of stationarity (2.6), (2.7). 


We apply for all the polynomial systems of equations the 
homotopy continuation approach as implemented in the 
PHCpack package m- For a brief introduction to the 
homotopy continuation method, see for instance HU. 


In the case of the systems of equations (2.4), (2.7), 
we decompose the three component complex vectors w ', 
w into real and imaginary parts. In turn we split every 
equation in the sets into real and imaginary parts. In this 
way, all indeterminants in all systems of equations have to 
be real, and we discard all nonreal solutions. Practically, 
we treat any solution as real if the imaginary part of any 
of the inderminants has an amount smaller than 0.001. 

Let us study some details of the potential in case 
it has the correct electroweak symmetry breaking 
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SU{2)l X U{1)y —>■ U{l)em- This is conveniently done 
in a new basis 


ip'.{x) = U,jipj{x), {1,2,3} (2.9) 

in which only the field >^ 3 ( 2 ;) gets a nonvanishing vac¬ 
uum expectation value VQjy/2. The unitary matrix U is 
determined by two rotation angles, which have to fulfill 

vi = sin(/32)sin(/3i)uo, 

V 2 = sin(/32) cos(/3i)uo, (2.10) 

V 3 = cos(/32)uo 


with / 3 i G [ 0 , 27 r[, (^2 G [ 0 , 7 r] and we have in particular 
^1+'^2+^3 = '^ 0 - In the new basis, the bilinear parameter 
vector (^q) from ( 1 . 11 ) becomes 


(a 


( ^ 

cos(2di)cos(d2)6 

0 

sin(/3i) cos(bi)(l -f cos^(/32))Ci + \/3/2sin^(/32)^8 
-cos(2di)sin(d2)6 
0 

sin(/32) cos(/32)(sin(2/3i)^i - v^^s) 


0 

\%/3cos(bi) sin(/3i) sin^(b 2 )Ci + i(l + 3 005 ( 2 ^ 2 ))^sj 

( 2 . 11 ) 

The correct global minimum has, in terms of these pa¬ 
rameter components, the potential value 


(V) = ^ (^0 - • (2.12) 

This result serves as a good cross-check of the numerical 
study. In particular, the potential value at vacuum has 
to be nonpositive in order not to lie above the unbroken 
minimum, which is given by a vanishing potential. 

The physical Higgs bosons follow from the diagonal- 
ization of the charged and neutral squared Higgs mass 
matrices. The neutral field of the third Higgs boson (/J 3 , 
that is, h^{x), is a mass eigenstate if the parameters fulfill 


^4 = ^5 = = 0 . 


(2.13) 


Therefore with a view on (2.11), we find the explicit con¬ 


ditions for the 0 ( 2 )( 8 )S 2 model, where the neutral compo¬ 
nent h^(x) is aligned with the vacuum expectation value. 
The mass squared of the neutral component is in this 


case 


~ 1 / 2 ^°^’ (2T4) 

Alignment thus requires that either vi = V 2 or = 0 
which is not the case, since the ratio of the two vacuum 
expectation values vi and V 2 is fixed by the ratio of the 
muon and tau masses, and a nonvanishing parameter 
is required in order to break the 0 ( 2 ) ® S 2 symmetry 
softly. 


Let us comment on the neutrino mixing angles ^13 = 0 
and 023 = 7 r/ 4 . Since these valnes seem not to be exactly 
fulfilled experimentally, we mention that deviations may 
be achieved by imposing further soft-breaking terms in 
the potential (1.6). We have seen that the value and the 
nature of the global minimum is rather sensitive to small 
changes of the potential. Therefore, we expect that defi¬ 
nite results would require a separate study of the changed 
potential. However, since additional soft-breaking terms 
do not affect the quartic part of the potential, and, in 
particular, we have found that for large parts of param¬ 
eter space stability follows from the quartic terms alone, 
we expect stability also in the respective cases of a po¬ 
tential imposing additional soft-breaking terms. 

We would like to mention that the parameters 02 , 03 , 
and 04 couple the neutral boson hP(x) with the other 
two generations of doublets. In general this may lead to 
changed phenomenology in Higgs boson production and 
decay of the h°(x) field. For some investigation of this we 
refer to [S]. A further detailed study for instance, with 
respect to a possible enhancement of the hP{x) decay rate 
into a pair of photons is left for future work. 


3. CONCLUSIONS 

The 0(2) 0^2 model [5] introduces three Higgs-boson 
doublets accompanied by an appropriate assignment of 
the elementary particles to irreducible representations 
of the group. In this way a neutrino mass matrix is 
generated which corresponds to mixing angles which are 
close to the experimental measurements. However, even 
though the symmetry restricts the model, the Higgs po¬ 
tential appears to be rather involved. Nevertheless, the 
recently introduced methods to study any three-Higgs 
doublet model m were applied to study the potential 
in detail. We have investigated stability, the station¬ 
ary points, and electroweak symmetry breaking of the 
Higgs potential by solving the corresponding station¬ 
ary equations, employing polynomial homotopy contin¬ 
uation. The method numerically hnds all the isolated 
complex solutions out of which we have extracted the 
physical real solutions. We have scanned over a range 
of values of the potential parameters. As expected, for 
too low values of the quartic parameters typically an un¬ 
stable potential is encountered. For parameter values, 
corresponding to a stable potential, the global minimum 
was detected. Our study reveals that in this model there 
are viable parameters corresponding to a stable global 
minimum with correct electroweak symmetry breaking. 
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